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Summary of Results: Peters (1992) reports that students in the intervention and control groups 

showed gains on the Orleans-Hanna test during the course of the school 
year (that is, from pretest to posttest). However, the test score gains of the 
two groups did not differ significantly. There was no evidence that the 
Saxon Algebra curriculum (intervention) was more or less effective than 
the University of Chicago Mathematics Project curriculum (control). 

Sample sizes were not adequate to allow for sufficiently precise estimates 
of the effect size. 






= Meets Evidence Standards 



O = Meets Evidence Standards with Reservations O 



= Does Not Meet Evidence Standards 



The What Works Clearinghouse (www.whatworks.ed.aov) was established in 2002 by the U.S. 
Department of Education’s Institute of Education Sciences to provide educators, policymakers, 
researchers, and the public with a central and trusted source of scientific evidence of what works in 
education. Please email all questions and comments to info(a)whatworks.ed.aov . The What Works 
Clearinghouse is administered by the U.S. Department of Education through a contract to a joint 
venture of the American Institutes for Research and the Campbell Collaboration . 
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Intervention: Saxon Algebra 

Operational Features 

The Saxon Algebra curriculum covers topics 
typically treated in a first-year algebra course. 
The Saxon curriculum teaches mathematics via 
“incremental development.” Incremental 
development as described by the developer 
introduces topics in small, understandable 
pieces (increments), permitting the assimilation 
of one facet of a concept before the next facet is 
introduced. All facets are then practiced 
together until another concept is introduced. 

The incremental introduction of topics is 
combined with continual review, wherein all 
previously learned material is reviewed in every 
lesson for the entire year. Topics are never 
dropped. Instead, they are increased in 
complexity and practiced every day, so that 
concepts will become more familiar over time. 

Instruction is typically teacher centered, with 
the instructor introducing the daily concept and 
demonstrating problems for the first 30 minutes 
of class time. Students, then, practice the 
problems during the remaining class time. 

In this study, one mathematics instructor was 
trained on both the Saxon Algebra curriculum 
and the curriculum from the University of 
Chicago Mathematics Project (UCMP). The 
teacher taught two classes of students, one using 
the Saxon Algebra curriculum and the other 
using the UCMP curriculum. Peters reports that 
the UCMP curriculum was designed to comply 
with the standards of the National Council of 
Teachers of Mathematics (NCTM). 

Thirty-six 8th-grade students were randomly 
assigned to two classes. The intervention group 
was taught using the Saxon Algebra curriculum, 
and the control group was taught using the 
UCMP curriculum. The study lasted from 
August 1991 to May 1992. 

People, Settings, and Timing 

The students in this study were enrolled in 8th 
grade and judged as “math talented” based on 
teacher recommendations, prior academic 
achievements, and personal maturity levels. 
Little other demographic information is 



reported. The Orleans-Hanna pretest was 
administered in August 1991, and the posttest 
was given in May 1992. 

Cost information 

Cost information is not reported in this study. 

Intended Duration 

Saxon Algebra curriculum is intended to be a 
complete and comprehensive full-year algebra 
curriculum. 

Scientific Rationale 

Peters (1992) does not present any particular 
scientific rationale for the study, other than to 
compare and contrast Saxon Algebra 
curriculum’s effectiveness with that of the 
UCMP curriculum, using the NCTM standards 
as the basis for comparison. Only the UCMP 
curriculum was based on the NCTM standards. 

Overview of the Study 

Purpose 

Peters sought to determine whether mathematics 
achievement differed between 8th-grade 
students taught in a Saxon Algebra curriculum 
classroom and those taught in a UCMP 
curriculum classroom. Peters was also interested 
in students’ overall satisfaction with the 
mathematics program. Thus, students were 
surveyed about the strengths and weaknesses of 
their experiences and about their comfort level 
with mathematics study. Students also 
participated in exit interviews with Peters, but 
this WWC Study Report focuses only on the 
achievement component of the study. 

Intervention Fidelity 

Peters reports little information about 
implementation. One teacher taught both the 
intervention and control classes. Peters reports 
that the teacher did not have any prior firsthand 
experience with the Saxon Algebra curriculum 
or the UCMP curriculum. Peters indicates that 
the instructor had read extensively about each 
program prior to the study. The teacher was 
provided with professional development 
opportunities for both curricula and was 
periodically monitored by the researcher to 
ensure treatment fidelity of key components of 
the Saxon Algebra intervention. 
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Outcome Measures 

Peters used two tests of mathematics 
achievement: the Orleans-Hanna Prognosis Test 
(a test of algebra readiness) and a set of four 
researcher-developed unit tests that were 
specifically aligned to content taught in both 
curricula. The items on the unit tests were 
reviewed by the researcher, the teacher, and 
other mathematics educators before the tests 
were administered. The Orleans-Hanna test has 
been documented as reliable and valid. No 
reliability or validity evidence was provided for 
the unit tests. 

Research Design 

The 8th-grade enrollment in the school totaled 
121 students, of whom 36 had been identified as 



“algebra ready” on the basis of California 
Achievement Test scores, prior school grades, 
and teacher recommendations. These 36 
students were randomly assigned to one of two 
classes: the Saxon Algebra curriculum class and 
the UCMP curriculum class. Analyses were 
conducted at the student level. 

Participant Fiow 

Peters notes that scheduling difficulties and 
student requests for other course offerings 
resulted in imbalanced groups: 19 students were 
in the Saxon Algebra class and 17 students were 
in the UCMP class. Peters does not document 
exactly how many students changed group 
assignments. (See Figure 1, Participant Flow.) 



Figure 1. Participant Fiow (Students) 



Random 

Allocation 



Intervention 

Allocation 



Purposeful Exclusions 
from Analysis: 

0 participants excluded 
from analysis 



Lost to Analysis: 

0 participants lost to 
analysis 



Analysis Sample 




Note. Intervention = Saxon Algebra curriculum; Control = University of Chicago Mathematics Project curriculum. 
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Reference Periods 

The study took place during the 1991-1992 
school year. The Orleans-Hanna pretest was 
administered in August 1991, and the posttest 
was given in May 1992. 

Baseline Data 

Using previous scores from the California 
Achievement Test and pretest scores from the 
Orleans-Hanna test, Peters did not find 
significant differences between the student 
groups. The numbers of boys and girls in each 
group are the only other student characteristics 
reported. (See Table 1.) 

Statistical Methods 

Using group assignment as a predictor, Peters 
presents analysis of variance results for the 
Orleans-Hanna posttest. Peters also provides an 
analysis of covariance, where scores from the 
pretest are used as a covariate to predict posttest 
scores. 



Outcomes and Estimation 

Students in the intervention and control groups 
showed gains on the Orleans-Hanna test during 
the course of the school year (that is, from 
pretest, August 1991, to posttest. May 1992). 
However, the test score gains of the two groups 
did not differ significantly. There was no 
evidence that the Saxon Algebra curriculum was 
more or less effective than the UCMP 
curriculum. (See Table 2 and Figure 2.) 

Table 2 presents the effects of the Saxon 
Algebra curriculum on mathematical 
achievement, as reported by Peters. This table 
presents unadjusted posttest means and standard 
deviations. Peters also reports the results of the 
surveys and exit interviews of the participating 
students; however, this WWC Study Report 
focuses only on the achievement results. 



Table 1. Pretest Characteristics of the Study Sampie 





Intervention (Saxon 
Algebra) 


Control (UCMP) 


California Achievement Test: 
Raw Mathematics Mean Score 


96.1 


94.9 


Orleans-Hanna Pretest: 
Mean Score (SD) 


89.8 (8.8) 


89.9 (5.6) 


Gender 


8 boys/1 1 girls 


8 boys/9 girls 


Note. UCMP = University of Chicago Mathematics Project curriculum. 




Table 2. Impact Reported by Peters (1992) 






Intervention (Saxon 
Algebra) 


Control (UCMP) 


Orleans-Hanna Posttest: 
Mean Score (SD) 


95.6 (4.5) 


95.1 (4.1) 



Note. UCMP = University of Chicago Mathematics Project curriculum. 
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Figure 2. Impact Calculated by Peters (2002)^: Orleans-Hanna 
Posttest 



110 




95.1 



0.5 



Saxon Algebra UCMP Control Mean Difference 

Intervention (n = 17) 

(n = 19) 



Note. Peters reports that the intervention group scores did not differ significantly from the control 
group scores (p > .05). UCMP = University of Chicago Mathematics Project. 

^Confidence intervals were calculated by the WWC. 



Intervention Developer Contact 
Information 

Saxon Publishers 

2600 John Saxon Boulevard 

Norman, OK 73071 

(800) 284-7019 

info @ saxonpublishers.com 

www.saxonpublishers.com 

Related Studies 

See reports on other studies of Middle School 
Math curricula. 

See reports on other studies of Saxon Algebra 
curriculum. 



Report Production 

Date created: October 28, 2004 

Topic area reviewed under: Curriculum-Based 
Interventions for Increasing K-I2 Math 
Achievement — Middle School 
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WWC study Ratings®: Peters (1992) 


Causal Validity: 


Meets WWC Evidence Standards, a Randomized Controlled Trial with No 
Randomization, Attrition, or Disruption Problems 


Participants were randomly assigned to intervention and control groups. The Orleans-Hanna Algebra Prognosis pretest was 
administered before the study at the beginning of the school year. Pretest scores did not show any significant differences 
between the study groups. The random assignment procedure was revised because of student scheduling conflicts, but the 
magnitude of the revision is unknown. No other extraneous events were identified that appeared to confound the 
intervention’s effects. 


Other Study 
Characteristics 


Study Rating 


Study-Specific Information 


Intervention Fidelity 


• • 


The Saxon Algebra curriculum is well designed and implemented and meets the 
definition for Middle School Math. Peters does not provide any evidence of changed 
expectancies, novelty effects, and disruptions or any information about implementation 
fidelity. 


Outcome Measures 


• • 


Two outcome measures were used in this study: the Orleans-Hanna Prognosis Test and 
researcher-developed unit tests in algebra. The Orleans-Hanna assessment shows 
evidence of acceptable reliability, and both outcome measures appear to be well 
aligned with the Saxon Algebra curriculum. 


People, Settings, and 
Timing 


• 


Although some important characteristics are represented in the sample, many are not. 
Students were selected from a single school, and the sample of students is part of the 
identified population. However, Peters documents variation in gender only and does 
not report variation in the location or academic track of classrooms in the study. 

The outcome measure was administered at the end of the school year. Peters reports 
that the study took place between August 1991 and May 1992. 


Testing within 
Subgroups 


• 


The intervention’s effect was tested across the entire sample but not within important 
subgroups. 


Analysis 


• 


Students were randomly assigned to intervention and control groups, but an unknown 
number of students changed group assignments because of scheduling conflicts and 
requests for other classes. In addition, the sample sizes were not adequate to allow for 
sufficiently precise estimates of the effect size. Peters does not report sufficient 
statistics for analyzing performances on the researcher-developed algebra tests. 


Statistical Reporting 


•• 


The statistical tests were adequately reported, and effect sizes could be estimated for 
the outcome measure of interest. 


Note. •• Fully meets criteria; • Meets minimum criteria; X Does not meet criteria. 

^ For more information on the criteria used to rate this studv, see the WWC Evidence Standards: Middle School Math. 
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